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/SBSTRiCT 


An  approach  to  the  analyses  of  discrete-valued  time  series  is  discussed. 

The  analyses  are  accomplished  in  the  spectral  domain  using  the  Walsh-Four ier 
transform  which  is  based  on  Walsh  functions.  This  approach  will  enable  an 
investigator  of  discrete  systems  to  analyze  the  data  in  terms  of  square- 
waveforms  and  sequency  rather  than  sine-waves  and  frequency. 

ir"' 

H(fe  developsja  general  signal -plus-noise  type  model  for  d i screte-va I ued 
time  series  in  which  Walsh-Four ier  spectral  analysis  is  of  interest.  .We  con- 
si  der^^the  problems  of  detecting  whether  or  not  a  common  signal  exists  in 
repeated  measures  on  discrete-valued  time  series  and  in  discrete-valued  processes 


collected  in  an  experimental  design.'^^iife  show>^|jfhat  these  models  may  depend  on 
unknown  regression  parameters  and  we  develop^consi stent  estimates  of  these  para- 
meters  based_on  the  finite  Walsh-Four ier  transform^  Applications  to  certain 
Markov  models  are  given,  however,  the  methods  presented  also  apply  to  non-Markov 
cases. 


1 


1.  INTRODUCTION 

Implicit  in  the  Fourier  (trigonometric)  analysis  of  time  series  is  one  of 
two  extreme  assumptions  about  the  process:  (1)  The  very  long  stretch  of  the 
time  series  is  the  only  time  series  we  want  to  consider  and  consists  of  the 
superposition  of  not  too  many  sinusoidal  terms  of  substantially  different 
frequencies;  and  (2)  The  time  series  is  to  be  regarded  as  a  reaiization  of  an 
eraodic  Gaussian  process.  It  is  one  of  many  possible  time  series  and  the  analyses 
are  directed  toward  the  properties  of  the  ensemble  of  the  series,  not  toward 
those  of  a  specific  realization  (cf.  Bril  linger  and  Tukey  (1982)). 

There  are,  however,  many  physical  situations  in  which  time  series  are  either 
positive  or  discrete  and  are  patently  non-normal  so  that  the  analyses  cannot  be 
handled  by  transforming  the  data  and  applying  Gaussian  techniques  (see,  for 
example,  Lewis  (1980),  p.  154).  Similarly  there  are  processes,  such  as  those 
which  take  values  in  a  discrete  finite  set,  which  can  neither  be  thought  of  as 
Gaussian,  nor  as  the  superpositions  of  well  separated  sinusoids.  Mode  is  for 
discrete-valued  time  series  which  have  an  ARMA  structure  are  considered  in 
Jacobs  and  Lewis  (1978  a,b,  1983)  and  in  Lewis  (1980).  For  the  case  of  continuous¬ 
valued  non-normal  time  series  it  is  perhaps  still  reasonable,  in  appropriate  cases, 
to  do  spectral  analysis  via  trigonometric  methods.  However,  in  the  cases  where 
time  series  take  values  in  a  discrete  (and  possibly  finite)  set,  it  makes  little 
statistical  sense  to  correlate  the  data  with  sines  and  cosines.  As  an  alternative, 
we  suggest  that  the  spectral  analysis  of  discrete-valued  time  series  be  accomplished 
in  the  "sequency"  domain  via  the  Walsh-Four ier  transform  (cf.  Ahmed  and  Rao  (1975), 
Kohn  (1980  a)  or  Morettin  (1981)).  This  seems  to  be  a  natural  alternative  to 
the  usual  Fourier  analysis  since  the  Walsh-Four ier  transform  is  based  on  the 
"square-wave"  Walsh  functions.  This  approach  would  enable  investigators  to  analyze 
discrete-valued  time  series  (which  we  may  think  of  as  square-waveforms)  in  terms 


of  square-waves  and  sequency  (switches  per  unit  time)  rather  than  sine-waves 
and  frequency.  As  empericaily  demonstrated  in  Beauchamp  (1975),  "the  respective 
roies  of  Walsh  and  Fourier  spectral  analysis  for  discontinuous  and  smooth- 
varying  waveforms  are  clear.  Where  the  signal  is  derived  from  sinusoidally- 
based  waveforms,  Fourier  analysis  is  relevant.  Where  the  signal  contains  sharp 
discontinuities  and  a  limited  number  of  levels,  Walsh  analysis  is  appropriate". 

The  Walsh  functions,  which  are  defined  via  the  Rademacher  functions  (cf. 
Ahmed  and  Rao  (1975),  Kohn  (1980  a)  or  Morettin  (1981)),  form  a  complete  ortho¬ 
normal  sequence  on  C0,1)  and  take  on  only  two  values,  -i-l  and  -1  (or  'on' 
and  'off').  They  are  ordered  by  the  number  of  zero-crossings  (or  switches) 
which  is  called  sequency.  Let  W(n,X),  n  =  0,1,2,...,  0  £  X  <  1,  denote  the 

n-th  sequency-ordered  Walsh  function,  then  W(n,»)  makes  n  zero-crossings  in 
C0,1).  The  first  eight  discrete,  sequency-ordered  Walsh  functions,  W(n,m/N), 

m,n  *  0,1 . 7,  corresponding  to  a  sample  of  length  N  »  2^  are  shown  in 

Figure  1.1  in  an  8x8  symmetric  matrix  called  the  Walsh  ordered  Hadamard 
matrix,  H^(3).  See  Appendix  A  for  details.  We  note  that  other  orderings  exist, 
for  example.  Pa  ley  order  and  Hadamard  order  are  often  used  (cf.  Ahmed  and  Rao 
(1975)),  however,  sequency  or  Walsh  ordering  is  more  natural  in  that  it  is 
comparable  to  the  frequency  ordering  of  sines  and  cosines.  We  will  discuss 
methods  of  generating  the  discrete  Walsh  functions  in  Appendix  A. 
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Hw(3)  = 


FIGURE  1.1;  Sequency-ordered  discrete  Walsh  functions,  W(n,m/N), 

n,m  =  0,1 . 7,  for  a  sample  of  size  N  =  2^  as  the 

rows  of  a  Hadamard  matrix. 


Walsh  spectral  analysis  has  been  used  for  several  purposes,  primarily  in 
the  Engineering  sciences,  such  as  speech  processing,  word  recognition,  image 
coding  and  transmission,  filtering  and  multiplexing.  It  has  also  been  used  to 
describe  biological  and  medical  systems  such  as  monitoring  EEG  and  ECG  signals. 
See,  for  example,  the  Proceedings  on  the  Applications  of  Walsh  Functions,  Ahmed 
and  Rao  (1975),  Beauchamp  (1975)  and  Harmuth  (1972)  to  mention  a  few.  Appli¬ 
cations  of  Walsh  functions  in  the  statistics  literature  are  rather  scarce  and 
we  mention  two.  Ott  and  Kronmal  (1976)  use  the  Walsh  transform  in  classification 
and  prediction  problems  for  strictly  stationary  binary  time  series.  Stoffer  and 
Panchalingam  (1985)  analyze  simulated  and  real  binary  time  series  in  the 
sequency  domain. 

At  present,  there  are  two  modes  of  development  of  Walsh  spectral  analysis 
in  the  literature.  The  first  mode  is  termed  Walsh  spectral  analysis  and  is 
developed  via  the  concept  of  dyadic  stat i onar i tv .  That  is,  it  is  based  on 
processes  {X(n);  n  *  0,1,2,...}  for  which  Cov(X(n) ,X(n  0  m) )  *  B(m)  is  a 


function  only  of  the  dyadic  distance  between  n  and  n  0  m,  where  n  0  m 

denotes  the  dyadic  addition  of  n  and  m  (cf.  Morettin  (1974  b,  1981)  for 

definitions,  discussions  and  references).  In  this  mode,  one  has  in  mind  that 

the  process  of  interest  is  the  superposition  of  not  too  many  Walsh  functions 

K 

of  substantially  different  sequencles,  that  is,  X(n)  =  I  Z(k)W(n,A.  )  where 

k=1  ^ 

Z( 1 ) , . . . ,Z(K)  are  uncorrelated  random  variables  with  mean  zero  and  variance 
2 

a.,  k  =  1 . K,  with  X, . X^  constants,  X.  X.,  i  ^  j-  The  other 

K  I  I  J 

mode  of  development  is  termed  Walsh-Fourier  spectral  analysis  and  is  based  on 
real  time  stationer ity.  Theoretical  results  concerning  the  statistical  appli¬ 
cation  of  Walsh-Fourier  spectral  analysis  are  relatively  recent  and  to  the  best 
of  my  knowledge,  are  limited  to  the  works  of  Kohn  (1980  a,b),  Morettin  (1974  a, 
1981,  1983)  and  Stoffer  (1985).  One  is  warned  that  in  the  examination  of  non- 
mathematical  literature  concerning  this  subject,  to  keep  in  mind  that  the  two 
different  modes  exist,  although  the  particular  mode  is  not  always  apparent. 

This  matter  can  be  quite  confusing  since  the  results  are  considerably  different 
and  results  from  one  mode  do  not  typically  apply  to  the  other.  An  excellent 
review  of  the  two  different  approaches  is  given  In  Morettin  (1981).  We  believe 
that  although  dyadic  time  has  some  theoretical  appeal  in  the  Walsh  spectral 
domain,  due  to  its  strange  behavior  (see,  for  example,  the  discussions  in 
Robinson  (1972)  and  Beauchamp  (1975)),  It  is  of  little  practical  use.  We  there¬ 
fore,  concentrate  on  real  time  stationary  processes. 

A  brief  account  of  the  existing  Walsh-Fourier  theory  as  well  as  some  new 
results  and  necessary  tools  to  be  used  in  the  sequel  are  given  in  the  next 
section.  In  Section  3  we  present  a  general  signal -plus-noise  type  model  for 
discrete-valued  time  series  in  which  Walsh-Fourier  spectral  analysis  is  of 
interest.  Next  we  consider  the  problem  of  detecting  whether  or  not  a  common 
signal  exists  in  repeated  measurements  on  discrete-valued  time  series.  It  is 
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shown  that  this  method  extends  to  d I screte-va I ued  time  series  collected  In  an  experi¬ 
mental  design.  Then,  we  consider  the  analysis  of  discrete  signal -plus-noise 
models  in  a  regression  setting  in  which  the  signal  is  observable  but  the 
process  depends  on  regression  parameters.  Our  main  goal  is  to  estimate,  via 
Walsh-Fourier  spectral  methods,  the  regression  parameters  and  the  noise  spectrum. 

In  all  cases  we  give  concrete  examples  by  showing  that  certain  Markov  chains 
satisfy  our  models,  however,  the  methods  presented  here  also  apply  in  non- 
Markov  cases.  For  completeness,  we  provide  a  discussion  in  Appendix  A  on  simple 
and  economic  methods  for  generating  the  discrete  sequency-ordered  Walsh  functions 
and  hence  the  finite  Walsh-Fourier  transform. 


Ml’,. 
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2.  PRELIMINARIES 

In  this  section  we  give  definitions,  estabiish  some  new  resuits,  and  give 
a  brief  summary  of  the  existing  Wa I sh-Four i er  theory  for  stationary  time  series 
which  we  use  in  the  sequel.  In  particular,  we  concentrate  on  those  properties 
which  we  can  directiy  appiy  to  discrete-valued  time  series.  Hence,  for 
examp ie,  we  do  not  consider  properties  of  processes  which  are  generalized  linear 
processes;  such  processes  can  be  handled  by  the  theory  given  in  Kohn  (1980  a,b), 
Morettin  (1983)  and  Stoffer  (1985).  At  present,  our  discussion  will  be  for 
univariate  time  series,  the  multivariate  versions  follow  in  an  obvious  way  and 
we  mention  them  briefiy  at  the  end  of  this  section. 

Let  X(0),  X( 1 ) , . . . ,X(N-1 )  be  a  sample  of  length  N  =  2^,  p  >  0  integer, 
from  a  weakly  stationary  time  series,  {X(n),  n  =  0,±1 ,±2, . . . },  with  absoluteiy 
summable  autocovariance  function,  =  ()ov(X(n) ,X(n+h) ) ,  h  =  0,±1,±2,...  . 

We  assume  for  now  that  the  constant  mean  value  of  X(n)  is  zero.  Let  W(n,X) 
be  the  n-th  Walsh  function  in  sequency  order,  and  let 

-1/2 

d^,(X)  =  N  y  X(n)W(n,X),  0  <  X  <  1  (2.1) 

N  n  ” 

n=0 

be  the  finite  (or  discrete)  Wa I sh-Four ier  transform  of  the  data.  The  logical 
covariance  of  X(n)  (cf.  Robinson  (19^2)  or  Kohn  (1980  a))  is  defined  to  be 

N-l 

T(j)  =  N  '  I  yU  ®  k  -  k) 
k=0 

where  by  j  ©  k  we  mean  the  dyadic  addition  of  j  and  k.  It  can  then  be 


shown  (cf.  Kohn  (1980  a))  that  the  variance  of  d|^(x)  is  given  by 


Taking  the  limit  (N  -*•<*>)  in  (2.2)  we  have  that  Var{d|^(X)}  -*■  f(A)  where 

0» 

f(X)  =  I  T(j)W(j,X),  0  <  X  <  1  (2.3) 

j=0 

is  called  the  Walsh-Fourier  spectral  density  of  X(n).  We  note  that  f(X) 
exists  since  the  absolute  summability  of  y^h)  implies  the  absolute  summability 
of  T(j).  Specifically,  Kohn  (1980  a.  Lemma  3)  shows  that  if 

lim  I  (1  -  I j|/2")  |Y(j)|  <  «  (2.4) 

n^  |j|<2" 

then  I  ^  well-defined. 

j=0 

If  X(0) ,X( 1 ) , . . . ,X(N-1 )  is  a  sample  of  length  N  =  2^,  the  finite  trans¬ 
form  (2.1)  is  calculated  for  X.,  =  m'N,  m  =  0,1,...,N-1.  Since  the  discrete 

N 

Walsh  functions  are  symmetric  in  their  arguments  for  N  =  2^,  that  is, 

W(n,m/N)  =  W(m,n/N),  m,n  =  0,1 . N-1,  (2.5) 

the  value  of  X^^  in  the  finite  Walsh-Fourier  transform  corresponds  to  sequency. 

As  with  the  usual  Fourier  analysis,  if  the  mean  of  the  series  is  unknown,  the 
only  sequency  of  the  form  X^^  =  m/N  for  which  the  transform  cannot  be  evaluated 
is  at  the  zero  (m  =  0)  sequency.  To  see  this,  let  9  =  EX(n),  all  n,  and 
note  that  for  m  =  0,1,..., N-1, 


where  6  is  the  Kronecker  delta.  See  Kohn  (1980  a.  Lemma  1).  It  is  clear 
from  (2.6)  that  the  mean  centered  transform  will  be  the  uncentered  transform 
except  at  m  =  0,  and  in  particular 

E{d,,(m/N)}  =  y  eW(n,m/N)  = 

n=0 

m  =  0.1,...,N-1. 

Kohn  (1980  a,  (^rollary  3)  gives  the  following  useful  results  on  the  con¬ 
vergence  of  the  second  moment  of  the  finite  Walsh-Fourier  transform  under  con¬ 
dition  (2.4).  Let  be  dyadically  rational  (that  is,  its  binary  representation 

is  finite).  If  X|^©X-»-0  as  N  =  2^-*'<»,  then 

E{d^(X..)}  -*•  f(X).  (2.7) 

N  N 

In  general,  the  asymptotic  covariance  of  the  Walsh-Fourier  transform  at  two 
distinct  sequencies  is  not  zero  (cf.  Kohn  (1980  a).  Theorem  5).  However,  if 
X^  and  X2  1^  afe  dyadically  rational  and  |x^  j^-X2  ^^1  i  N  ^  with  X.  O  X  -►  0, 
i  =  1,2  as  N  =  2^  -*■  «,  then 

Various  authors  have  established  central  limit  theorems  for  the  finite 
Walsh-Fourier  transform  under  a  wide  range  of  conditions  (cf.  Kohn  (1980  a), 

Morettin  (1983)  and  Stoffer  (1985)).  Wo  state  three  versions  which  are  applicable 
to  d i screte-va I ued  time  series.  The  first  version  (Assumption  2.1)  follows  its 
trigonometric  counterpart  given  in  Hannan  (1973)  and  can  be  found  in  Kohn 
(1980  a.  Theorem  4).  The  second  version  (Assumption  2.2)  follows  its  trigono¬ 
metric  counterpart  based  on  the  existence  of  higher  moments  given  in  Bril  linger 


(1975)  and  can  be  found  in  Morettin  (1983,  Theorem  1).  We  remark  that  the 


above  two  versions  exist  side  by  side  and  that  neither  is  included  in  the 
other  (cf.  Morettin  (1983)).  The  third  version  (Assumption  2.3)  is  similar  to 
Stoffer  (1985,  Theorem  3.1),  however  it  Is  sufficiently  different  to  warrant  a 
proof  which  is  provided  in  Appendix  B.  This  version  establishes  a  central 
limit  theorem  for  discrete-valued  second-order  stationary  processes  which 
satisfy  a  type  of  finite  dependence  property. 

ASSUMPTION  2.1;  X(n)  is  strictly  stationary  with  zero  mean.  Let  be  the 

a-field  generated  by  {X(j),  j  ^  n};  put 

a.  =  CE{E(X(n)  |  F^_  J  -  E(X(n)  |  j  >  0, 

00 

F  is  trivial  and  T  ot .  <  ». 

j=0  J 


ASSUMPTION  2.2;  X(n)  is  strictly  stationary  with  zero  mean  and  finite  moments. 

Let  C^( , . . . , j^)  =  Cum{X( ) , . . . ,X( j^) }  be  the  r-th  cumulant  of  X(n), 

00  00 

f  •  •  • » =  0,±1  ,±2, . . .  ;  'I  ...  |C^(  , . . . ,  — ^)|  <  ®. 

Jl=0 


ASSUMPTION  2.3;  X(n)  is  second-order  stationary  with  zero  mean  and  covariance 

2+6 

function  Y(k).  sup^E{|X(n)|  }  <  «  for  some  6  >  0.  Let  F^  be  the  a-field 

generated  by  {X(j),  j  <.  n}.  There  exists  a  positive  integer  k  such  that 

(i)  E{X(n)  IF  }  =  0  a.s.,  and  (ii)  E{X(n)X(n+k)  IF  }  =  Y(k)  a.s.  for 

'  n-tc  '  n-K 

k  =  0,1,...  ,k:-1  . 

If  condition  (2.4)  and  either  Assumption  2.1,  2.2  or  2.3  hold,  then  d  (X) 
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converges  in  distribution  to  a  normal  variate  with  mean  zero  and  variance  f(A) 
given  by  (2.3).  We  note,  for  example,  that  d i screte-va I ued  time  series  which 
are  based  on  mixtures,  such  as  the  discrete  MA  processes  described  in  Jacobs 
and  Lewis  (1978  a,b,  1983),  Lewis  (1980),  and  the  geometric  processes  discussed 
in  Langberg  and  Stoffer  (1985),  satisfy  the  dependence  properties  of  Assumption 
2.3. 

In  order  to  be  able  to  consistently  estimate  the  Walsh-Fourier  spectrum, 

we  need  asymptotic  results  for  smoothing  the  Walsh-Fourier  periodogram, 

2 

l.,(X.,)  =  d.|(X.,).  We  now  state  the  following  results  in  a  theorem  for  use  in 
N  N  N  N 

the  sequel.  The  theorem  combines  the  three  previous  central  limit  theorems 
with  a  theorem  given  in  Kohn  (1980  a.  Theorem  4). 

THEOREM  2«t;  Let  condition  (2.4)  and  either  Assumption  2.1,  2.2  or  2.3  hold. 

Let  Xj  *  j/N,  1  <.  j  1  N-1,  and  suppose  for  N'‘**’^j(M)  N^' 

. "  l*J(i).N'>J(k).Nl  i 

-d  ^  k,  £,k  =  1 . M.  Then  d.,  -*•  N(0,A)  where  d.,  =  (d,,,(X.,,.  •  .d.,(X  .,))' 

-N  -N  N  J(1J,N  N 

and  A  is  an  M  x  M  diagonal  matrix  with  f(X)  along  the  diagonal.  Also 

d'4  2  f(X)xS  (2.8) 

-1  2 
so  that  M  d.'.d.,  is  an  estimate  of  f(X)  having  variance  2f  (X)/M. 

*N-N  ^ 


If  we  let  M  -►  »  as  N  «  with  M/N  0  in  Theorem  2.1,  the  smoothed 
periodogram  M  is  a  mean  square  consistent  estimate  of  the  Walsh-Fourier 

spectrum  f(X),  0  <  X  <  1. 

Results  obtained  for  the  univariate  finite  Walsh-Fourier  transform  carry 
over,  in  an  obvious  way,  to  the  vector  case,  say  X(n)  =  (X^ (n) , . . . ,X^(n) ) ' ,  except 
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that  (cf.  Kohn  (1980  b).  Section  3) 


(1)  The  logical  covariance  will  now  be  r  x  r  matrices  given  by 

N-1 

T(j)  =  N  I  {r(j  e  k  -  k)  +  r'(j  e  k  -  k)}  (2.9) 

k*0 

where  r(h)  is  the  r  x  r  autocovariance  matrix  of 
X(n)  =  (X^(n),...,X^(n))». 

(2)  The  Walsh-Four ier  spectrum  f(X)  is  an  r  x  r  real  positive 
semidefinite  matrix. 

(3)  Cov{l.  .(X)l,  (X)}  f.p(X)f  .  (X)  +  f.  (X)f».(X)  (2.10) 

I  j  -cm  i-c  jm  im  tj 


as  N  -*•  «  where  l..(X)  is  the  (i,j)-th  element  of  the 

^  J 

X  X' 

periodogram  matrix  with 

X  -1/2 

dJJ(X)  =  N  }((n)W(n,X). 

n=0 


r  X  r 
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3.  SIGNAL-PLUS-NOISE  MODELS  FOR  DISCRETE- VALUED  TIME  SERIES 

In  this  section  we  discuss  models  for  discrete  systems  in  which  Walsh- 
Fourier  analysis  is  desirable.  We  consider  a  discrete  version  of  the  signal- 
plus-noise  models  used  for  sinusoidal  and  for  Gaussian  processes.  In  general, 
write  the  r  x  1  vector,  discrete-valued  time  series  as 


JJ(n)  =  §(n)  +  e(n),  n  =  0,±1  ,±2, . . .  (3.1) 

where  S(n)  is  a  random  stationary  discrete  signal  which  possibly  depends  on 

unknown  parameters  0  =  (6, ,...,6  )',  and  e(n)  is  a  zero-mean  discrete- 

1  q 

valued  process  (possibly  white  noise)  which  is  uncorrelated  with  S(n).  We 

note  that  the  support  of  X(n),  S(n)  and  e(n)  need  not  necessarily  be  the 

same  and  that  there  may  be  some  dependence  structure  between  §(n)  and  |(n). 

For  a  specific  example  of  such  a  process,  consider  a  macro  model  on  a 

finite  state  space  (cf.  Basawa  and  Prakasa  Rao  (1980)).  Let  X.(n),  j  =  1,...,r 

J 

denote  the  number  of  individuals  in  state  j  at  time  n.  In  particular,  X.(n) 

J 

Is  the  aggregate  over  several  independent  chians  evolving  simultaneously.  Let 

0  .f  j  =  1 . r,  be  the  probability  of  being  in  state  j  at  any  given  time, 

J 

and  let  Q  denote  the  total  number  of  individuals  under  consideration.  Denote 

the  r  X  1  vector  X(n)  »  (X^(n) . X^(n))'  and  suppose  that  X(n)  is  Markov 

with  transition  probabilities  p..,  Il'.Jir.  Then,  for  n  =  1,2,..., 

i  J 

r 

EX.(n)  =  J  EX.(n-1)p,  .,  ^  j  <.  r, 

J  |ai  '  'J 

from  which  we  obtain  the  signa I -plus-noise  model 
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r 

X.(n)  =  y  X.(n-1)p.  .  +  e  .(n),  1  <  j  <  r,  (3.2) 

where  g(n)  =  (e ^ (n) , . . .  ,e^(n) ) ' ,  n  =  0,1,2,...,  is  a  zero-mean  multinomial- 

type  white  noise  process  and  is  uncorrelated  with  the  random  signal 

r 

S(n)  =  (S,(n) . S  (n))*,  S.(n)  =  7  X.(n-1)p.  .,  1  <  j  <  r.  To  see  that 

'  r  J  i=l  '  iJ  “  “ 

e(n)  and  S(n)  are  uncorrelated,  first  consider  one  individual  at  a  time  and 
let  Y.  (n)  =  1  if  individual  q  is  in  state  j  at  time  n,  and  Y.  (n)  =  0 

j.q  j.q 

otherwise,  1<.q<.Q,  lljl*".  Denote  the  corresponding  error  term  by 

r 

w.  (n),  that  is,  write  Y.  =  7  Y.  (n-1)p.  .  +  w.  (n).  Then  for  1  <  q  <  0, 

J.q  J.q  >1  '.q  ^J  J.q 

j ,  k  =  1 , . . . ,  n , 


Cov{w.  (n),Y^  (n-1)}  =  Cov{Y.  (n)-  7  Y.  (n-1)p,.,Y,  (n-1)} 

J.q  k.q  J.q  ,.q  "ij-  k.q 


Noting  that  aggregate  of  the  independent  individual  error  terms, 

e.(n)=  y  w.  (n), 

I  ^1/1 


J.q 


and  that  S  .(n)  is  the  aggregate  of  the  independent 


we  see  that  e(n)  and 


q=1  Q  . 

individual  signal  terms,  S.(n)  «  I  Y.  (n-1)p.., 

J  q=l  i  =  i  'J 

S(n)  are  uncorrelated.  Hence,  we  have  exhibited  a  model  of  the  d i screte-va I ued 

signal -plus-noise  form  given  in  (3.1)  where,  in  this  case,  the  signal  S(n)  is  a 
2 

function  of  the  r  transition  probabilities,  p.  .,  i,j  =  1,...,r. 

^  J 

For  the  signal -plus-noise  model  presented  in  this  section,  Walsh-Fourier 
analysis  would  be  useful  for  detecting  whether  or  not  a  discrete  signal  exists 
in  a  given  system,  and  if  so,  determine  the  cyclic  behavior,  in  terms  of  sequency, 
of  the  signal.  Moreover,  for  discrete  systems  in  which  the  signal  's  observable 
but  the  process  depends  on  unknown  parameters,  Walsh-Fourier  methods  can  be  used 
to  consistently  estimate  the  parameters  as  well  as  to  consistently  estimate  the 


error  spectrum.  We  discuss  these  types  of  analyses  in  the  following  subsections. 


3.1  Detecting  a  Cowwon  Signal 

Now,  consider  the  problem  of  detecting  whether  or  not  a  common  discrete¬ 
valued  signal  exists  in  Q  replications  of  a  discrete-valued  time  series 

{X^(n)},  q  =  1 . Q  which  are  of  the  signal-plus-noise  form  (3.1).  Many  of 

our  techniques  follow  those  of  their  trigonometric  counterparts  developed  by 
Bril  linger  (1973;  1975;  Section  7.9;  1980). 

As  an  example,  suppose  we  wish  to  analyze  Q  independent  queueing  situations 

hypothesized  to  be  similar.  Let  X  (n),  q  =  1,...,0.  be  the  number  of 

9 

individuals  in  queue  q  at  time  n  and  suppose  that  EX^(n)  =  6^  for  all  n. 

Let  S(n)  be  a  zero-mean  stationary  discrete- valued  signal  hypothesized  to  be 

common  to  all  queues,  such  as  a  rate  of  change  in  the  number  of  individuals  in 

a  queue,  and  let  e  (n),  q  »  1,...,Q  be  stationary  zero-mean  discrete-valued 

9 

processes  with  common  Wa Ish-Four ier  spectrum.  Whether  or  not  a  common  signal 
exists  among  the  Q  queueing  situations  is  expressed  in^ whether  or  not  the 
signal,  or  equivalently  its  Walsh-Four ier  spectrum  is  identically  zero. 

In  general,  we  suppose  that  the  discrete  system  is  of  the  form  X^(n), 
q=l,...,Q;  n  =  0,l,...,N-l,  N  =  2*’,  p>0  integer,  and  can  be  modeled  as 

X  (n)  =  9  +  S(n)  +  e  (n)  (3.3) 

9  9  9 

where  are  constants,  S(n)  is  a  realization  of  a  stationary  discrete-valued 

time  series  with  mean  zero,  and  e  ^a),  q  =  1,...,Q,  are  independent  reali- 

9 

zations  of  a  zero-mean  discrete-valued  stationary  time  series  which  is  un¬ 
correlated  with  S(n).  Let  Y  (b),  h  *  0,±1,±2,...,  denote  the 

autocovariance  functions  of  S(n)  and  eq(n),  1  i  9  i  Q,  respectively.  We 


assume  that  and  satisfy  condition  (2.4)  and  that  every  linear 

combination  of  S(n)  and  e^(n),  )<.<?<.  0»  satisfies  either  Assumption  2.1, 

2.2  or  2.3.  Denote  the  logical  covariances  of  S{n)  and  e  (n),  1  <  q  <  Q 

q  -  - 

by  T-_(j)  and  t  (j),  respectively,  and  the  respective  Walsh-Four ier  spectrum 
Od  c  c 

of  S(n)  and  e  (n),  1  <  q  <  <J,  by  f^,K(X)  and  f  (X),  0  <  X  <  1.  The 

q  ~  ~  SS  ee  “ 

following  conditio! ns  hold: 


(1) 

EX  (n)  = 

0  , 

q 

q 

(2) 

X 

q  q 

(3) 

fy  X  <x) 

=  f.Ax)  +  f  (X), 
So  ee 

q  q 

(4) 

"x  x/J^ 
q  ^ 

=  Tgs^j).  q  ^  1, 

(5) 

^x  x.^^^ 
q  ^ 

=  fss^A).  qr<^. 

Before  we  proceed  with  the  analyses  we  need  the  following  lemma. 


LEI*<A  3.1 :  Let  {X^}  and  {Y^}  be  sequences  of  random  variables  on  a  probability 

space  (n,F,P)  such  that  for  all  n,  EX^  <  »  and  EY^  <  ».  If  X  ^  X, 

n  n  n 

Y  ^  Y,  EX^  EX^,  and  EY^  EY^  as  n  •,  then  EX  Y  -►  EXY. 
n  n  n  n  n 

PROOF:  By  Chung  (1974,  Theorem  4.5.4)  the  conditions  EX^  <  •,  X^  ^  X,  and 

EX^  -►  EX^  imply  the  uniform  i ntegrab 1 1 i ty  of  {|X|^}.  Similarly,  {IyI^}  is 
uniformly  i ntegrab I e.  By  the  Cauchy-Swartz  inequality. 


for  any  set  A  e  F  and  hence  by  Chung  (1974,  Theorem  4.5.3),  the  uniform 

2  2 

integrabi  I  ity  of  }  and  ^  Imply  the  uniform  integrabi  I  ity  of 

{lx  Y  1}.  Clearly  X  Y  ^  XY  and  EX  Y  <  »,  all  n,  hence  by  Chung  (1974, 
n  n  n  n  n  n 

Theorem  4.5.4),  EX  Y  EXY. 

n  n 

The  particular  analyses  of  this  section  are  based  on  the  following 
theorem. 


THEOREM  3. 1 ;  Let  {X^(n);  q  =  1 . (J;  n  =  0,...,N-1}  satisfy  the  model 


conditions  (3.3).  Let  X  m  =  1,...,M  be  defined  as  in  Theorem  2.1 

j(m),N 


Then  the  finite  Walsh-Four ier  transform  d.,  (X.,  .  ,,) 

N,q  j(m),N 


•1/2 


N-1 


1  1  <1  1  Qf  has  the  representation 


\  ®  u  +2  a.s. 

N,q  j(m),N  m  q,m 


(3.4) 


as  N  «  where  the  u  are  independent  N(0,f-«(X))  variates  and  the  z 

m  Sq  q,m 


are  independent  N(0,f^^(X))  variates.  Moreover,  u  and  z  are  mutually 

» £  m  q ,  m 


independent,  l^m^M,  lldlQ. 


PROOF;  Taking  the  finite  Walsh-Four ier  transform  of  (3.3)  we  obtain 


‘*N,q''' j(m)  ,N'  '^N^^j(m),N^  *  *^N,q'" j(m)  ,N' 


d..  _(X„_,  .,)  =  d::(X,,_,  ..)  +  d^.  JX.,_,  ,,),  1  <  q  <  Q 


where  d^^*)  and  df.  (•)  are  the  We  I sh-Four i er  transforms  of  S(n)  and  e  (n), 
N  N,q  q 


respectively.  The  constant  term  drops  out  in  view  of  (2.6)  and  the  fact  that 


Xj(m)  ^  Invoking  Skorokhods  Representation  Theorem  (Skorokhod  (1956))  and 


Theorem  2.1,  d;,(X .  .,)  and  df,  (X.,  . 

N  j(m),N  N,q  j(m),N 


)  have  the  representations 
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C  £ 

-*•  u  a.s.  and  d.,  (X.,  .  ■*■  z  a.s.  as  N  -►  «  where  the 

N  j(m),N  m  N.q  j(m),N  q.m 

u  are  independent  N(0,f-c^X))  variates  and  the  z  are  independent 
m  SS  q,m 

N(0,f  (X))  variates.  It  remains  to  show  that  u  and  z  are  mutually 

ee  m  q,m 

independent  and  since  they  are  jointly  normal,  it  suffices  to  show  that  they 
are  uncorrelated.  Since  S(n)  and  e  (n)  are  uncorrelated  we  have  that 

q 

E{df,(X.,  .  .,)d^  (a.,  .  .,)}  =  0  for  ail  N,  m  =  1 . M,  and  q  =  1 . Q. 

N  j(m),N  N.q  j(m),N 

Since  Y  satisfy  condition  (2.4)  we  have  by  (2.7) 

So  ee 

E{d^(X  ..  .  ^)^}  N  1  £  m  £  M, 

N  j(m),N  SS  N.q  j(m).N  ee 

>af.  <A  E{u_2.  .) 


1  <  q  <  0.  Thus  by  Leh»a  3.1  «>  have  E<<'n<»j(„) .N’^N.q'* j(m) ,N 

as  N  -»•  «  and  hence  E{u  z  }  =  0.  m  =  1 . M.  q  =  1 . . . .  ,0. 

m  q.m 


m  q.m 


Using  the  representation  (3.4)  ^e  may  proceed  with  the  problem  of  determining 
whether  or  not  a  common  discrete- valued  signal  exists.  Our  analysis  follows 
that  of  the  analysis  of  random  effects  models  (cf.  Scheffe  (1959)).  Consider 
the  quantities 


1  KI^  =  P  ’  I 

N.*  j(m).N  ^  N. 

q-’ 


M 


,  (X  .  1  <  m  <  M 

N.q  j(m).N  -  - 


N.*  j(m).N 


(3.5) 


m=1 


0  M  , 

I  I  cd^,  (X.,  , ,,)  -  d^,  (X .  jy 

q=l  tn»1  N.*  j(m).N 


(3.6) 


By  Cochran's  Theorem  and  in  view  of  the  representation  (3.4).  the  quantities 
(3.5)  and  (3.6)  have  the  almost  sure  representations 


A 


my  y 

I  (u  +  z  =  Cfcc<^5  +  ^  (^)/q:ix;:  a.s. 

tn  ‘.m  SS  ce  M 

in=  I 


(3.7) 


Q  M  2  7 

I  I  (2  „  -  z  „)  =  f  (X)x^,«  ,,  a.s. 

q^l  q.*"  •.m  «  M(Q-1) 


(3.8) 


respectively,  as  N  -►  »,  where  the  x  variates  in  (3.7)  and  (3.8)  are 

2 

independent.  The  fact  that  the  X  variates  are  independent  follows  from  the 

fact  that  Covlz  -z  ,z  )  =  0. 

q,m  •  ,m  •  ,m 

The  test  of  the  null  hypothesis  that  there  is  no  common  signal,  S(n)  =  0 
or  equivalently  examined  for  parlicular  sequencies  X  by 

compar i ng 


Q  T  dj  (X.,  ,  ^)/M 
N,-  j(m),N 


9  M  2 

1*1  rn^l  j(m).N  N,*  j(m),N  ^ 


(3.9) 


with  an  M(Q-l)  <^'str ibution.  In  view  of  (3.7)  and  (3.8),  (3.9)  has  the 
representation 


f,,(X) 


^M,M(Q-1) 


as  N  -►  «». 


Using  the  representations  (3.7)  and  (3.8)  it  is  easy  to  find  an  asymptotically 
(N  -►  oo)  unbiased  estimate  of  the  Walsh-Fourier  spectrum 

A 

observed  signal,  say  f_-(X),  0  <  X  <  1.  Denoting  the  numerator  of  (3.9)  by 
Q  ?cs(X)  and  the  denominator  of  (3.9)  by  f  (X)  we  see  that  the  desired 

AA  ee 


quantity  is 


fcc(^>  =  fnn(X)  -  q’’  f  (X).  (3.10) 

Ow  C£ 

We  note  that  due  to  the  invariance  properties  of  maximum  likelihood  estimates 
(MLEs)  and  the  fact  that  the  asymptotic  distributions  of  the  transforms  are 

A 

Gaussian,  the  estimate  (3.10)  is  also  the  bias  corrected  MLE  of 

fgs^X)  is  an  asymptotic  (N  <»)  sense. 

Next,  we  consider  a  one-way  design.  Suppose  now  that  L  different  treat¬ 
ments  are  replicated  on  experimental  units  and  stationary  discrete-valued  time 
series  are  recorded  for  each  of  the  units.  For  example  suppose  L  treatments 
are  given  to  groups  of  patients  (Q  patients  in  each  group)  who  suffer  from 
a  particular  allergy,  and  we  record  the  number  of  allergic  reactions  per  day  of 
each  patient  for  N  days.  The  data  collected  will  not  only  be  discrete  count 
data,  but  for  many  allergies  it  is  mostly  on  the  order  of  0,1, 2, 3  and  very 
seldom  above  this.  Hence,  a  Wa I sh-Four i er  as  opposed  to  trigonometric  de¬ 
composition  of  the  observations  would  be  more  useful.  This  type  of  design  is 
similar  to  the  balanced  one-way  random  effects  design  considered  in  Scheffe 
(1959,  Section  7.2)  and  in  Brillinger  (1980,  Section  4.2).  Specifically,  for 
q  =  1,...,Q,  !,  -  1,...,L,  n  =  0,1,...,N-1,  N  =  2^,  p  >  0  interger,  let 


V‘"’  “  V  *  V"'  v‘"’ 


(3.11) 


where  e  „  are  constants;  S.(n)  are  L  independent  stationary  discrete-valued 
94  4 

time  series  with  mean  zero,  logical  covariance  Walsh-Four ier 
spectrum  independent  realizations  of  a  zero-mean 
discrete- valued  stationary  time  series  with  logical  covariance 


20 


Wa Ish-Fourier  spectrum  in  the  model  (3.3),  S^Cn)  and  ^^(n) 


are  uncorrelated,  q  *  1 . Q,  however  Sj(n)  and  3^®  nnutually 


independent  for  j  ^  and  q  =  We  assume  that  every  linear  com¬ 


bination  of  and  ~  ^ . 0*  satisfies  either  Assumption  2.1, 


2.2  or  2.3,  t  =  1,...,L.  Under  the  model  assumptions  we  have 


(1)  EX  .(n)  =  0 

ql  ql 


(2)  Tj 

(3)  fj 

(4)  Tj 

(5)  f. 


q^' 

y 

'V 

T„(J). 

V  (X) 

=  f,.l^(X)  + 

f  (X), 

ql' 

SS 

ee 

ql' 

X 

V 

■  "ss'J'' 

q  p 

■  i 

,X 

q  p. 

^l 

It  is  seen  that  the  coherence  between  any  two  observations  in  the  same  treatment 
i  s 


+  f  (X)3. 
SS  SS  ee 


This  corresponds  to  the  intraclass  correlation  coefficient  discussed  in  Scheffe 
(1959). 

Taking  Wa Ish-Fourier  transforms  in  (3.11)  and  following  Theorem  3.1  we  have 
the  representation 


dll  =  u.  +  z  »  a.s. 

N,q£  j(m),N  tm  q^ 


(3.12) 


as  N  -►  00,  ms  1 . M,  where  the  u^  are  independent  N(0,f22^)^^^  variates 


and  the  are  independent  N(0,f^^(X))  variates  which  are  independent  of 


the  u^.  all  q,  t  and  m.  Using  the  representation  (3.12)  we  consider  a  test 
for  determining  whether  or  not  there  is  a  treatment  effect,  that  is,  whether 
or  not  S^(n)  =  0  for  t  =  1,...,L,  and  all  n.  Equivalently,  we  test 
whether  or  not  ^55^^^  -  0*  0  <  X  <  1.  We  may  now  follow  the  one-way  random 

effects  analysis  given  in  Scheffe  (1969,  Section  7.2).  Define 


^N,*-e^^j(m),N^ 


^-l 


Q 

I 

q=1 


j(m) ,N^ ' 


1  <  m  <  M 


M  ^  ®  •-  '  I  \  M^*  1  <  m  <  M 

N,**  j(m),N  N,*-C  j(m),N  - 


and  appropriate  sums  of  squares 


M  L 


Q  I  I  Cd^,  Ax,,  ,  .,) 

it]  j(m),N 


d„  (X.,  , 
N,**  j(m),N 


(3.13) 


M  L  Q 

J,  ’-^N,q^^^j(m),N^  “  *^N,  j(m)  ,N 


,(X 


):2. 


In  view  of  (3.12),  (3.13)  and  (3.14)  have  the  almost  sure  representations 


M  L 


Q  I  1  +  z  -  -  u  -  z  ) 

^  A.  hn  •Im  ^m  ••m 


m= 1  £= 1 


•m 


and 


M  L 


I  l  l  <2  ,  -  Z  .  >■ 

m.1  ril  ,.l 


(3.14) 


(3.15) 


(3.16) 


respectively,  as  N 


We  may  carry  this  representation  one  step  further  and 


represent  (3.15)  and  (3.16)  as 


CQ  fjjcx)  .  =.s. 


(3.17) 


and 


f 

ee 


(X)x 


2 

ML(Q-I) 


a.s. 


(3.18) 


2 

as  N  ->•  «,  where  the  X  variates  in  (3.17)  and  (3.18)  are  independent. 

The  test  of  the  null  hypothesis  ^55^^^  =  ^  follows  easily  in  view  of 

(3.17)  and  (3.18).  For  a  particular  sequency  X,  compare  the  statistic 


M  L 


Q  I  I  Cd.,  p(X..  ,  .,)  -  d,,  (X.,  ,  .,)r/M(L-1) 

m=1  l=\  j(m),N  N,**  j(m),N 

15  L  Q  Z 

I  I  I  J  -  d,,  »(X.,  ,  JJ  /ML(Q-l) 

N.q-C  j(m),N  N,*£  j(m),N 


with  an  F^(l-1).ML(Q-1)  distribution. 

To  estimate  the  Walsh-Fourier  spectrum  of  the  signal,  ^^^(X),  one  may  use 
representations  (3.15),  (3.16),  (3.17)  and  (3.18).  That  is,  the  expected  value 
of  (3.15)  divided  by  M(L-l)  (cf.  3.17)  is  Q  foc^^^  ^  (X)  and  the  expected 

value  of  (3.16)  divided  by  ML(Q-I)  (cf.  3.18)  is  f  (X).  This  is  similar  to 

EE 

the  previous  case  (cf.  3.10)  and  one  may  proceed  as  was  done  there. 

As  suggested  in  Bril  linger  (1973,  1980),  it  is  helpful  to  graph  the  above 
statistic  as  a  function  of  sequency,  X,  and  to  further  indicate  the  null  significance 
on  the  graph.  However,  in  doing  so,  one  must  keep  in  mind  that  the  asymptotic 
(N  -V  «)  covariance  of  the  Walsh-Fourier  transform  at  tw^  distinct  frequencies 
is  not  necessarily  zero  in  contrast  to  the  trigonometric  case. 


In  this  section  we  concentrate  on  the  discrete  s igna l-p I us-noi se  model 


where  the  signal  is  observabie  but  the  vector  process  of  interest,  X(n), 


n  =  0,1,..., N-1,  depends  on  unknown  regression  parameters.  At  the  end  of 


this  section  we  apply  the  results  of  this  section  to  estimating  the  transition 


matrix  in  a  Markov  model.  We  suppose  that  we  may  write  the  model  (3.1)  in  the 


r  X  1  vector  form 


X(n)  =0+3  S(n)  +  e(n). 


(3.19) 


where  6  is  an  r  x  1  vector  of  constants,  S(n)  is  a  q  x  1  vector. 


observable  d i screte-va I ued  signal,  0  is  an  r  x  q  matrix  of  regression  para¬ 


meters,  and  e(n)  is  a  d i screte-va I ued  zero-mean  white  noise  process  which  is 


uncorrelated  with  the  signal  S(n).  Let  denote  the  autocovariance 


matrix  of  S(n),  where  I”  ^  Further,  we  assume  that  every  linear 


combination  of  the  components  of  e(n)  satisfy  either  Assumption  2.1,  2.2  or 


2.3.  Let  T__(j)  and  t  (j)  be  the  logical  covariance  matrices  of  S(n) 

O  w  C  £ 


and  e(n),  respectively  (cf.  2.9)  and  note  that  t  (j)  =  0  for  j  ^  0  where 
-  EE  r 


0  is  the  r  X  r  matrix  of  zeroes.  Let  1<.a,b£q, 

r  ob  b  » 1 

Q  D 


be  the  q  x  q  positive  definite  Walsh-Fourier  spectrum  of  S(n)  and  let 


f  (A)={f  (X)},  1<a,b<r,  be  the  r  x  r  Walsh-Fourier  spectrum  of 

£0  £  f€,  "" 

a  D 


e(n) . 


Taking  transforms  in  (3.19)  and  following  the  previous  section  we  have  the 


representation 


"N  j(m),N  -N  j(m),N  -m  -a.s. 


(3.20) 


where  0  (1)  denotes  a  vector  variate  tending  to  the  zero  vector  almost 

-a.s. 

surely  as  N  ^  z^  are  independent  multivariate  N(Q,f^^(X))  variates; 

Y  -1/9  e  -1/9  ^ 

^  K.)  =  N  I  X(n)W(n.X.,  ,  J  and  d,,(X  =  N  Y  S(n)W(n,X 

-N  j(m),N  -  j(m),N  -N  j(m),N  -  j(m),N 

n=u  n=u 

denote  the  r  x  1  and  q  x  1  finite  Walsh-Four ier  transforms  of  the  vector 
time  series  X(n)  and  §(n),  respectively. 

If  we  assume  that  M  q+r,  we  may  consider  the  asymptotic  problem  of 
estimating  the  parameter  matrix  g  and  the  error  spectrum  in  the  multivariate 
analysis  setting  (see  Anderson  (1958),  Chapter  8).  Similarly,  tests  of  hypotheses 
about  the  elements  of  6  may  be  carried  out  by  MANOVA  techniques  with  an 
appropriate  partitioning  of  that  matrix  (cf.  Anderson  (1958),  Section  8.3). 

Following  least  squares  theory  in  view  of  (3.20),  the  estimate  of  B  as  a 
function  of  sequency  is  given  by 

I  ^  ^  I  ^  r  ^  (3.21) 

M,N  -N  j(m),N  ~N  j(m),N  ~N  j(m),N  •'N  j(m),N 


and  the  corresponding  estimate  of  the  error  spectrum  'S 


f”’^(X)  =  m"’  I  Cd?i(X.,  ,  .,)  -  L  ^,(X)d^(X  K,)3i:i)J(X  J 

E,€  "N  j(m),N  M,N  *N  j(m),N  *N  j(m),N 


M,N  -N  j(m),N 


(3.22) 


Before  discussing  the  consistency  of  the  estimates  .,(X)  and  f  ’  (X), 
we  state  the  following  results  which  follow  from  the  model  assumptions  and  the 
results  of  Section  2.  First,  denote 


i!»:i 


XfW 


vS 

I 


m 


•iNl 


^  m-i  ‘^N,a^^j(m),N^‘^N.b^^j(m),N^'  ^  i  a»b  <  q 


(3.23  a) 


f  (A)  =  M  ^  V  ^  M^'  1<a<r,  l<b<q 

N,a  j{tn),N  N,b  j(m),N  -  -  ’  _  _  '< 


(3.23  b) 


(i)  =  m"'  ”  d‘  (X 
S-S  m.l  '*•» 


y  d  (x  .,  .  ..Jd..  .(X.,  .  .,),  1  <  a.b  <  r 

N,a  j(tn),N  N,b  j(m),N  _  ' 


(3.23  c) 


where  d^.  (•)  and  df,  (•)  denote  the  finite  Walsh-Fourier  transform  of  the 

N,a  N,a 


a-th  element  of  §(n)  and  £(n),  resoectively.  Then  as  M,N  -►  *  with  M/N  -*■  0 


we  have 


(1)  E  f'  c  <X)  -*•  fc  c  (A), 
^a'^b  ^a’^b 


(2)  E  f  (X)  f  (X), 
^a’^b  ®a’®b 


(3)  E  (X)  =  0, 

a  b 


(4)  M  Var{f5’']'5  (X)}  -  2Cf3  g 
^a’  b  *a’*b 


(5)  M  VarCf”'*^  (X)}-'-2Cf  (X)D^, 

®a'^k  ®a*Ek 

a  b  a  b 


(6)  M  Var{f”’"  (X)}  -V  f  (x)f^  .(X), 

^a’^b  S’"a 


It  follows  from  the  above  that  as  M,N  -*•  »  with  M/N  -►  0,  f.’  -  (X)  -*•  f  _  (X), 

^a’^b  ^a’^b 


•-  •  «_  ■  •  ^  ^  1  ■ 
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M  N  M  N 

f  ’  (X)  -►  f  (X),  and  f  *  -  (X)  ■+•  0  in  mean  square  and  hence  in  pro- 

e  ,e.  e_»ek  e 

a  D  a  D  a  0 


babi I ity. 

We  now  show  the  consistency  of  the  estimates  (3.21)  and  (3.22) 


THEOREM  3.2;  Let  the  model  assumptions  (3.19)  be  satisfied.  Then  as  M,N  -►  ® 


2M.N, 


with  M/N  -*■  0  the  estimates  6^.  given  in  (3.21)  and  f  '  (X)  given  in 

M  t  N  £€ 


(3.22)  are  consistent  in  probability  for  6  and  f  (X),  0  <  X  <  1, 

ee 


respectively. 


PROOF;  Using  the  definitions  of  (3.23),  write 


®  +  f”'J(X)Cf”''^(X)]“’ 

M,N  e,S  So 


(3.24) 


u  ,,M,N  .  xM.N.  .  ..M,N  ... 

where  f  _(X)  =  {f  e  and  f--  (X)  »  {f.  ^  ane  r  x  q  and  q  x  q 

e,S  SS  S^,S 

a  b  a  b 


matrices,  respectively.  By  previous  results  we  know  that  as  M,N  -►  ®  with 


M/N  -►  0,  f^'^(X)  ^  fec^^^  which  is  positive  definite,  and  f^’p(X)  S-  o 

SS  SS  ^  e,S  rxq 


where  0  is  an  rxq  matrix  of  zeroes.  In  view  of  (3.24)  we  see  that 
rxq  ^ 


.,(X)  is  consistent  for  6. 
M,N 


:m,n. 


To  show  the  consistency  of  f^^  (X)  given  by  (3.22),  first  note  that  we 


may  write  the  r  x  r  matrix  f^’^(X)  =  (X)}  whose  elements  are  given 

^a'^b 


by  (3.23  c)  as 


,M.N  .  ..-1 

ee 


M 


M  '  I  CdJ;(X.,  ,  ^)  -  6  df,(X.,  .  ,.):CdJ5(X.,  ,  .,)  -  8  dS(X  ,  ^)]'. 
'N  j(m),N  'N  j(m),N  -N  j(m),N  -N  j(m),N 


M,N^ 


Since  f  (X)  is  consistent  for  f  (X),  it  suffices  to  show  that 
ee  ee 


f^’*^(X)  -  f^’*'^(X)  -►  0  in  probabi  I  ity  as  M,N  »  with  M/N  -►  0.  To  see  this 
ee  ee 


write 


M 


N  M  N  - 1  »  y  ^  * 

f  -  f  ’  (X)  =  M  '  ^  ^  m^cb  -  e„  .(X):' 

ee  ee  “N  j{m),N  -N  j(m),N  M,N 

fn®  I  ** 


M 


+  C6  -  L  ^.(x):  m“'  I  d^(x.,  ,  „)d^'(x.,  ,  .,) 

M,N  ^  -N  j(m),N  -N  j(m),N 


m=1 


M 

+  cL  ^.(x)  -  61  M“’  I  d^(x.,  ,  ^)d^'(X.,  ,  J  6' 

M,N  -N  j(m),N  -N  j(m),N 


m 


in=  I 


We  consider  the  first  term  of  (3.25),  the  other  cases  will  follow  by  similar 


(3. 


methods.  Expanding  d^^CX^^^j  l,^)  we  have 


M 


M,N, 


m=1 


.  f^-'‘<X)C6  -  5„^„(X)3'  .  0,. 


in  probability  as  M,N  -►  <»  with  M/N  -►  0  by  (3.24)  and  the  convergence  results 

-  ,M,N  .  . 

for  f_-  (X)  and  f  »  (X). 

SS  eS 


We  close  this  section  with  an  application  to  Markov  chains.  In  particular, 

consider  the  macro  model  given  by  (3.2)  and  suppose  that  we  are  interested  in 

estimating  the  transition  probabilities  p.  ..  We  shall  assume  that  detailed 

I J 

transitions  are  not  available,  otherwise,  the  p.  .  may  be  estimated  by  MLE 

^  J 

techniques,  see  Basawa  and  Prakasa  Rao  (19B0,  Section  2.1).  Recall  that  the 


model  is  of  the  form 


X^(n)  =  I  +  e^(n),  1  <  -£  <  r,  (3.26) 

k=1 

where  X^(n)  is  the  (aggregate)  number  of  individuals  in  state  t  (over  several 

independent  Markov  chains)  at  time  n  and  ^^^n)  is  zero-mean  white  noise 

which  is  uncorrelated  with  Xj^(n-I)  for  k  =  1,...,r,  and  r  is  the  number 

of  states.  We  suppose  that  data  Is  available  in  the  form  {X^(n),  Z  =  l,...,r; 

n  =  0,...,N-1}  and  that  the  total  number  of  individuals  in  the  system,  viz. 
r 

^  X.(n),  is  fixed.  To  avoid  singularities  we  remove  X  (n)  from  the  model 
(3.26).  Let  T  be  the  total  number  of  individuals  in  the  system,  then  for 
Z  —  l,...,r— 1, 


r-I  r-i  r-1 

X„(n)  «  T(1  -  I  P.p)  +  I  CX.(n-l)  +  [  X  (n-1  ):p ..  +  e«(n) .  (3.27) 

^  j=1  j=1  J  k=1  ^ 


r-1  r-1 

Now,  put  0fl  =  T(1  -  I  p.»),  S.(n)  =  X.(n-l)  +  I  X  (n-1)  with  S  .(0)  =  0, 

j=1  J  J  k=1  J 

and  let  X(n)  =  (X,(n),...,X  ,(n))',  0  =  (9, . 0  ,)', 

-  1  r-1  -  1  r-1 

S(n)  =  (S^ (n) , . . .  ,S^_.|  (n) ) '  and  e(n)  =  (e^  (n) , . . .  ,e^_^  (n) )  ’ .  We  have  already 
seen  that  g(n)  and  S(n)  are  uncorrelated.  In  view  of  (3.27)  the  model 
(3.26)  may  now  be  represented  in  the  form  of  (3.19),  that  is 


X(n)  =  9  +  P  S(n)  +  e(n),  n  ^  1,  (3.28) 


where  P  is  the  (r-1)  x  (r-1)  matrix  of  unknown  transition  probabilities 


29 


'•’•V 


i 


1 


m 


11 

*^21 

Pr-1.1 

12 

• 

”22 

Vi, 2 

• 

• 

1.r-1 

‘'2.r-l 

-  Pr-1,r-1 

^j(m),N' 

m  =  1 , . 

^j(m),N  * 

way  that  M  2(r-l)  we  obtain  the  consistent  estimate  of  the  transition  matrix 


P(X)  *  (A)Cfgg  (X>D  , 


.her.  I  3S<»j,„,,N>4'“j(»).N>' 


fj5  (X)  •  M  J  -N‘‘j<m).N’4  ‘‘jCmX.N* 


y  *  1  /9  ' 

v  ^  Ki^  =  N  y  X(n)W(n,X.,  ,  J  and 
-N  j(m),N  -  j(m),N 

n*u 


c  -1/7  '*■’ 

*'  n*!  k*1  '' 


In  this  case  we  also  obta i n  a  cons i stent  est i mate  for  9 ,  name  I y , 


e,(X)  =  T(1  -  I  p  .«(X)),  I  =  1 . r-1, 

^  i=l 


where  'S  the  j£-th  element  of  P(X).  The  consistent  estimate  of  the 


error  spectrum  i s 
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^  M  N  ^  M  N  ^ 

K  =  fyi  (X)  -  P(X)f"’"'  P'(X). 

e ,  e  XX  SS 


The  actual  Implementation  of  this  estlamtion  procedure  is  then  based  on  the 
methods  described  in  the  Appendix  A. 
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APPENDIX  A 


In  this  appendix  we  discuss  the  computation  of  the  finite  Waish-Fourier 

transform  for  samples  of  length  N  *  2^,  p  >  0  integer.  The  discrete  Waish 

functions  are  calculated  via  the  Hadamard  matrix,  H(p),  which  is  defined  to 

be  the  symmetric  orthogonal  N  x  N  matrix  whose  (u,v)-th  element,  u,v  =  0,1,..., N-1, 

P“1  u.v. 

is  equal  to  ♦(u,v)  »  H  (-1)  where  the  binary  representations  of  u  and 

j=0 

V  are  given  by  (u  ,tU  , . u.)  and  (v  ,,v  v.),  respectively, 

*  p-1  p-'t  0  p-i  p-2  0 

u.  and  V.  are  either  0  or  1.  For  example,  with  N  =  8,  the  (1,5)  element 
J  J 

of  H(3)  is  (>(1,5)  =  (-1)^^^^^  a  -1  whereas  the  (3,7)  element  is 
(>(3,7)  =  (-1)^'*’'''’^  =  1.  The  Hadamard  matrix  gives  the  discrete  Walsh  functions 
as  rows  (or  coiumns)  in  what  is  called  natural  or  Hadamard  ordering.  To  obtain 
the  Walsh  functions  in  sequency  order,  we  can  reorder  the  rows  of  H(p)  according 
to  the  number  of  sign  changes.  We  denote  the  sequency  or  Walsh  ordered  Hadamard 
matrix  by  H^(p).  An  alternate  method  of  obtaining  H(p)  uses 

"bit-reversal  Gray  code"  to  rearrange  the  rows,  however  this  is  essentially  the 
same  technique  as  counting  the  sign  changes,  see  Ahmed  and  Rao  (1975)  for 
details.  Since  these  approaches  involve  counting  sign  changes,  they  are  not 
very  efficient  procedures.  We  shall  discuss  a  fast  Walsh-ordered  Hadamard  trans¬ 
form  in  a  moment.  The  Hadamard  matrix  can  be  generated  recursively,  H(0)  =  1, 
and 


H(k+1) 


H(k) 

H(k) 

H(k) 

-H(k) 

,  k  *  0,1,2,.., 


so  for  example 
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H(1) 


and  H(2) 


1111 
1-1  1-1 
1  1-1-1 
1-1-1  1 


Denote  H(2)  =  ChQ(2) ,h^ (2) ,h2(2) ,h2(2)D  where  tJjtZ),  i  =  0,1, 2, 3  is  the 
i-th  column  of  H(2),  then  the  corresponding  Walsh-ordered  Hadamard  matrix  is 
H^(2)  =  ChQ(2)  ,132(2)  ,hj(2)  ,j3^ (2)3.  The  procedure  of  obtaining  the  Walsh-ordered 
Hadamard  transform  from  its  definition  either  requires  storage  of  the  Hadamard 
matrix,  or  recomputation  whenever  the  elements  of  sre  needed.  Hence, 

either  the  sample  length  is  restricted  to  about  p  =  10  or  20,  or  the  procedure 
is  extremely  slow.  There  are,  however,  fast  methods  which  can  reduce  the 
number  of  computations  (additions  and  subtractions)  by  about  2^  Vp  times  less 
than  by  using  the  definition.  The  method  we  discuss  here  may  be  found  in  Ahmed 
and  Rao  (1975,  Chapter  6)  who  also  gives  a  computer  subroutine.  The  Walsh- 
Hadamard  matrix  can  be  computed  as 

P 

H„(p)  »  n  H.(p)  .  B  (AD 


n 

’51 


fj!' 

I 

i 


4 


I  I 
s  s 


I  -I 
s  s 


I  -I 
s  s 


J 


and  1^  being  the  s  x  s  identity  matrix.  The  matrix  B  in  (Al)  is  a  matrix 
which  bit-reverses  the  order  of  the  data.  For  example,  with  N  =  2^  the 
bit-reversal  of  1  =  (0,0,1)  is  4  *  (1,0,0),  and  the  bit-reversal  of 
3  =  (0,1,1)  is  6  *  (1,1,0)  so  that  X(1)  is  exchanged  with  X(4)  and  X(3) 
is  exchanged  with  X(6)  in  the  data  vector.  !f  X  =  (X(0) , . . . ,X(N-1 ) ) '  is 
the  data  vector,  N  *  2*’,  the  fast  finite  Wa Ish-Four ier  transform  is  computed 


d^(A^)  =  H.(p).B  X 


where  *  (0/N, 1/N, . . . , (N-1 )/N) ' .  For  example,  if  N  =  2  ,  the  Walsh-ordered 


Hadamard  matrix  can  be  decomposed  as 


H^(3)  . 


ol  Tf 


'J  L 


where 


,  V. 
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n 

i 


4 

'4 


1 


m 


i 


*  w 
*■<1 


i 

w 


>5&’' 

li’ 


I 


2  ,1/2  -1/2  ^  ■>2,1/2 
^  ^^^''kT+l  "•  ^ 


<  n"’^^  I  E{|X(kT+j)|^)’^^ 


from  which  we  obtain 


E{v^  1^}  <  r^N~\(0)  <  kV\(0) 


and  hence  the  desired  result  follows.  In  a  similar  manner  we  can  show  that 


E{u^  1^}  £  TN  'k^y^O)  £  k  V^y(O) 


to  obtain  the  desired  result  for  u,  ... 

k,N 


To  establish  a  central  limit  theorem  for  N 


Y.  we  use  a  martingale 
j=1 


central  limit  theorem  found  in  Brown  (1971,  Theorem  2).  We  shall  follow  Brown's 

T 

notation.  Let  k  be  fixed  and  let  S,  -  =  J  Y.  ,  :  recall  that  F  is  the 

k,T  j,k  n 

a-field  generated  by  {X(j),  j  £  n}.  Then,  {S.  f.-r_  '•  T  =  1,2,...}  form 

K  p  I  K  I  *IC 


a  martingale.  To  see  this  note  that 


^^^k,T+1  I  ^kT-K^  “  ^j,k  ^^^T+1,k  ^  ^kT-x^’ 

J 


The  terms  in  are  {X(n)W(n,A),  n  =  kT  +  j;  j  =  1 ,2, . . .  .k-ic}.  We 


are  given  that  E{X(kT+j)  |  =  o  a.s.  and  since  ^ 


^  1  J  i  we  have 


E(X(kT.J)  I  .  EE(X(kT.J>  |  |  F^^_^  =  0  a.s. 
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I 


i 


«9 

a 


and  hence  E{Y_  ,  j  F  __  }  =  0  a.s.  as  was  to  be  shown.  Following  Brown  (1971), 

I  T  I  f  K  K  I  “K 


^.T  =  ^CJ-D-k’  =k.T'^''k.r  ''  typical  term  in 

I<^.m<k-k,  l<j<T.  Let 


n^  =  (j-1)k  +  £  and  =  (j-1)k  +  m,  then  by  hypotheses  of  the  theorem, 


E{y  y  I  F  }  =  W(n,  ©  n  ,X)E{X(n  )X(n  )  |  F.  .  } 

n^  n2  (j-l)k-ic  12  12'  (j-l)k-K 


=  W(n^  ©  n2.X)Y(n2~n^ )  a.s. 


=  E{y  y_  }. 


"l  "2 


y  2  2^ 

Hence,  V  _  =  J]  EY.  =  EV^  _  =  s.  _  a.s.  The  application  of  Theorem  2  of 

k,l  ..  j  k,i  k,l 

J 

Brown  (1977)  entails  showing  that  as  T  ■»•  », 


(ii)  s"^,  I  E{Y^  J^Y.  j  >  es.  t 
k,T  .  .  j,k  J»k  ~  k,T 
J 


)}  -*•  0 


for  any  c  >  0,  where  !(•)  is  the  indicator  function.  We  have  already 


established  (i)  and  we  proceed  with  establishing  (ii).  First,  we  show  that 


sup  E{|X(n)|^^^}  <  «>  for  some  5  >0  implies  that  sup.  E{|Y.  < 

n  '  '  J 


Let  n^  =  (j-l)k  +  t  and  a(n^)  =  W(n£,X).  Then  by  Minkowski's  inequality 


E{|Y.,k|^^S^'‘'^  =  E{|a(n^)X(n,)  +  ...  +  a(n^_^)X(n^_^) 


2+6, 2+6 


£  I  E{|X(n,)|^'^^}^'^^ 
£=1 


£  (k-tc)Csup^  E{  |X(n) 


I 
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so  that  for  any  j. 


E{|Y.  sup  E{|X(n)|^'^^}  <  « 

j  •  I'  n 


as  was  to  be  shown.  Following  a  similar  argument  to  the  one  above,  we  can 

2  2 

show  that  EY.  ,  is  uniformly  bounded,  and  since  EY .  ,  does  not  vanish  as 

J,k  j,k 

2 

j  ^  00  we  see  that  s.  ^  =  0(T).  Also,  one  can  show  that  since 

k  9 1 

sup.  E{|Y.  .  I  }  <  00  for  some  6  >  0,  there  exists  a  random  variable  Y. 
j  j  ,K  k 

2 

with  EY,  <00  such  that 
k 


for  all  j  and  all  u  >_  0.  See  Stoffer  (1985,  Lemma  3.2).  Thus, 


I,  i  «.,T»  i  ^  ”k.T»  *  “ 


as  T  -»■  00.  Hence  by  the  Brown  (1971)  mar+ingale  central  limit  theorem  we  have 

1.  t"’/2  j  y2  ,.,1/2_-1/2,  -1/2  I  ^ 

that  for  fixed  k,T  T  )N  converges  in 

j=1  j=l 

law  to  a  normal  variate  whose  variance  depends  on  k.  as  T  -►  “»  or 
equivalently  N  -►  oo.  Clearly  then,  for  fixed  k,  (N^^^T  ^^^k  '^^)N  7  Y^  . 

j=’ 

converges  in  law  to  a  normal  variate  as  N  -►  «».  Letting  k  -*■  oo  establishes 
the  desired  result. 
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